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Overview 


>Who, what is the College Board? 

>Vision of Feedback on College Readiness 
>The Research & Development Agenda 
>Some Preliminary Results 
>Q&A 
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Overview of Education Bn the United States 


> Federal, state and local government have varying degrees of authority in education. 

• US Department of Education does not control what schools do. 

• Control is decentralized to states and local educational agencies. 


s The primary control is found in local education agencies (often school districts) controlled by local boards of education who are 
elected officials representing the community. 

• Districts acquire funding from states and local taxes. 

s The amount of control over local education agencies varies by state and prescribed by state laws. 

• For example, Connecticut has more control over local education agencies than Texas. 


> In 2008-2009, it was projected that there were 74.1 million students enrolled in schools (primary, 
secondary, post-secondary levels) 

• Private schools comprise approximately 14% of enrolled students in the U.S. 

> In 2007-2008, there was $972 billion in federal, state, & local budgets for education. 

• In primary and secondary schools, only 9% of the revenue in public schools comes from the federal government (2006- 


• In primary and secondary schools, the average expenditure per pupil was $9,1 00 (2006-2007). 

> In 2005-2006, 

• there were over 97,000 primary and secondary schools. 

• there were over 4,200 post-secondary institutions. 

• 66% of high school completers enrolled in post-secondary institutions. 

> The unique position of the College Board as a private, not-for-profit organization permits us to cut across 
state borders to offer assessments and educational initiatives for secondary and post-secondary schools. 


Sources: U.S. Census Bureau (2009). The 2009 Statistical Abstract: The National Data Book. U. S. Census Bureau (www.census.gov). U.S. National Center for Education 
Statistics, Digest of Education Statistics, annual, and Projections of Education Statistics, annual 


2007). 
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The College Board 


• Not-for-profit membership organization founded in 1900: 

• 5,400 members representing schools, districts, colleges/universities, and other 
educational organizations 

• The College Board is governed by 31 -member Board of Trustees with guidance from 
three national assemblies and six regional assemblies. 

• Mission: Connect students to college success and opportunity 

• Serves 7 million students and their parents, 23,000 high schools, and 3,500 
colleges through major programs and services in college admissions, 
guidance, assessment, financial aid, enrollment, and teaching and learning. 

• Assessments: ACCUPLACER, Advanced Placement Program (AP), College Level 
Examination Program (CLEP), PSAT/NMSQT and SAT. 

• Educational Initiatives: AP Summer Institutes, CollegeEd, College Board Schools, 
Excelerator Program, Florida Partnership, Exam Readiness Program, Professional 
Development, and Springboard. 

• Advocacy Efforts: National Office of School Counselor Advocacy, Financing Higher 
Education 

• Services: College Guidance Services, College/University Enrollment Management 
Services, Student Search Service 
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Vision in Developing, Validating and 
Reporting on College Readiness 


The College Board is focused on the integrated use of information from our assessments in providing educators 
with multiple indicators about student college readiness to help them in working with students. 


College Board Standards for College Success 
State Standards 


Secondary 


Post-Secondary 



Annually, 50% of the high school graduates in the U.S. have taken the SAT with 
participation rates at states ranging from a high of 1 00% to a low of 3%. 
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The phases in developing and validating the 
indicators of college readiness 


1. Understand the conceptual context from the literature 

> Articulate the conceptual framework of predictors of college success 

> Understand the context 

2 . Develop the metrics 

> Develop indicators to represent college readiness 

> Examine statistical properties and empirical relationship with other variables 

3. Validate the metrics using college outcomes 

> Empirically link the college readiness indicators with college outcomes. 

> Develop benchmarks 

4. Produce reports to secondary level educators 

> Represent the college readiness indicators in score reports 

> Use empirical methods to develop 

5. Evaluate the use of the information provided in the reports 

> Empirically examine how the information is used by educators 
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Predictors of College Success 


College Skills 

Content Knowledge 
Achievement 

Non-Cognitive 

Personal Qualities, 
Experiences, 
Characteristics 

School Characteristics 
& Context 

Guidance 

Verbal Reasoning 

Math 

Motivation 

Letters 

Grades 

Career Interests 

Math Reasoning 

Language Arts 

Follow-through 

Essay 

GPA 

Study Skills 

Writing 

Science 

Communication 

Community Service 

Weighted GPA 

Interest in Major 

Metacognition 

Social Studies/ Humanities 

Conscientiousness 

Extra-curricular 

Rank 

Self Efficacy 

Creativity 

Foreign Language 

Leadership 

Work Experience 

Courses Completed 

Aspirations 

Practical Knowledge 

Language Proficiency 

Other Personality 

Literacy in Second Lang 

Academic Rigor 

Realistic Self-concept 

Spatial Relations 



Teacher Ratings 

AP/Honors Courses 


Intellectual Curiosity 



Gender 

School Size 





Ethnicity 

School Quality 





Residence 






Age 






Family Education/ Income 






Ability to Pay 






Ability to Benefit 




Source: Camara, W. J. & Kimmel, E. W. (2005). Choosing Students: Higher Education Admissions Tools for the 21 st Century. Mahwah, NJ: Lawrence Erlbaum Associates. 
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College Success 


> Research has shown that 3 factors account for academic 
success: 

1. HS grades 

2 . HS courses 

s The number of courses (4 vs 3 years of math) 
s The highest level completed (Calculus vs Algebra II) 
s The rigor of the courses (honors, AP vs standard) 

3. College Admissions Tests 

> Studies by Cliff Adelman, Education Trust, National Center of 
Educational Statistics and other groups have been widely cited 
and accepted by policymakers. 

■ References offered at the end 

> But... no one has developed a comprehensive set of metrics 
that report on these 3 metrics for students and schools, YET 
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College Readiness Conceptualization 
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What is the SAT? 


Since its launch in 1926, the SAT helps college admissions officers make fair and informed admissions 
decisions. 

The SAT tests the subject matter learned by students in high school and how well they apply that 
knowledge — the critical thinking skills necessary to succeed in college. 

The SAT is offered seven times a year in the United States and six times at international sites. It last three 
hours and 45 minutes. Consists of 10 separately timed sections: 

• Three sections test critical reading (70 minutes total) 

• Three sections test mathematics (70 minutes total) 

• Three sections test writing (60 minutes total) 

• One variable (unscored) section tests critical reading, mathematics, or writing (25 minutes total) 

The SAT assesses critical thinking and problem solving skills in three areas: 

• Critical reading 

• Mathematics 

• Writing 

The SAT includes three kinds of questions: 

• Multiple-choice questions 

• Student-produced responses (mathematics only) 

• Essay question 

The SAT Questionnaire contains more than 100 questions on demographic information course taking, 
course performance, aspirations, educational goals, self ratings of ability and extra-curricular activities, 
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Why Do Colleges Value the SAT? 


• Along with high school grades, the SAT is the best 
predictor of college success. 

• Helps them overcome the challenges of unequal 
opportunities, variable standards and grade inflation. 

• Provides a valid, nationally consistent measure of 
what students have learned and how well they apply 
that knowledge. 

• Helps colleges match the right student with the right 
institution to maximize student success. 
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Admissions officers in colleges use a 
variety of indicators for college entrance 

• Strong majority of college admissions officers say that test 
scores are of “considerable or moderate” importance in their 
decisions 

• 94% Grades in college prep courses 

• 92% Grades in all courses 

• 90% Strength of curriculum 

• 89% Admissions test scores 

• 67% Class rank 

• 64% Essay / writing sample 

• 61% Recommendations (counselor /teacher) 

• 52% Extracurricular activities 

• 52% Student’s demonstrated interest 


Source: NACAC Admission Trends Survey, 2007. Arlington, VA: National Association of 
College Admissions Counseling (www.nacacnet.org). 
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Computing the Academic Intensity Index 


> Using information collected from students about 
their academic courses taken in high school (SAT 
Questionnaire) develop scales by discipline - 
preliminary and draft! 


- English (0 to 5) 

✓ Zero was awarded if a student did not take at least 3 years of English 
^ Points added if more honors and AP courses taken 

■ Math (0 to 5) 

s Zero was awarded if a student did not take at least Algebra 1 by 8 th grade 
s Points added if more specific types of math courses taken and honors/AP 

■ Science (0 to 5) 

s Zero was awarded if a student did not take any science. 
s Points added if more specific types of science courses taken and honors/AP 

■ Social Science (0 to 5) 

v' Zero was awarded if a student did not take at least three or more courses in social science 
s Points added if more specific types of math courses taken and honors/A 


Source: Wyatt, J. & Wiley, A. (December, 2008). Preliminary Results of an Academic Intensity Scale. Unpublished technical report. New York: The College Board. 
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SAT SAT 


Some Preliminary Results of the Academic 
Intensity Index 




Source: Wyatt, J. & Wiley, A. (December, 2008). Preliminary Results of an Academic Intensity Scale. Unpublished technical report. New York: The College Board. 

rvfl 
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Some Preliminary Results of the 
Academic Intensity Index 



Source: Wyatt, J. & Wiley, A. (December, 2008). Preliminary Results of an Academic Intensity Scale. Unpublished technical report. New York: The College Board. 
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Benchmarks for the Academic Intensity Index 

• Conduct regressions to identify the best 
combination of predictors for different scaling 
options (FGPA, persistence). 

• Do these separately for each indicator 

• Compute the % of students meeting different 
benchmarks (by ethnicity, SES, language) to 
estimate impact. 

• Use a policy capturing approach (or standard 
setting) with external educational experts to 
determine final levels. 
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SAT Benchmark Study (2007) 


• Based on old SAT 

• Class of 1995 (41 institutions 
representing 167,000 
freshmen) 

• Logistic regressions utilized to 
predict probability of getting a 
FGPA of C (2.0) andTGPA of 
B- (2.7). 

• Settled on 65% probability for C 
(2.0) and B- (2.7) 

• Two benchmarks at 800 and 
1180. 

• We will utilize similar approach 
in establishing benchmarks for 
each of three indicators. 



400- 500 600- 700- 800- 900- 1000- IT CO- 1200- 13CO- 1400- 1500- 

500 600 700 800 900 1000 1100 1200 1300 1400 1600 1600 


SAT Total Score (V+M) 


Source: Kobrin, J.K. (January, 2007). Determining SAT Benchmarks for College Readiness. RN-30. New York: The College Board. Web link: 
http://www.collegeboard.com/research/pdt7RN-30.pdf 
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A new SAT benchmark study 


• Use the new SAT validity data from 1 1 8 colleges to generate data for a benchmark 
study. This involves determining apriori the % of students in the sample (and cohort) 
that would be college ready at each probability level (50%, 60%, 65%...) and each 
criterion level (C, C+, B-, B...) 

• Next, determine likely impact of various benchmarks by comparison to other metrics 
and by examining results based on college selectivity and for underrepresented 
students. 

• Identify an appropriate panel to make decisions about CR and train them in the 
decisions that need to be made. 

• These will be policy experts in college readiness, primary, secondary, and post-secondary education. 

• These will not be pure content experts but college readiness experts. 

• Bring experts together for a one-day benchmark setting exercise at a selected site 
and ask for recommendations. 

• Final approval made by the College Board based on the panel’s recommendations. 

• Publication and Dissemination of these college readiness indicators and 
benchmarks -> August 2009. 
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College Readiness Reports 


1. Three metrics: 

• SAT benchmark 

• Academic Intensity benchmark 

• HS Grades (GPA adjusted or not adjusted by school) 

2 . Compensatory Model will provide an overall metric of college 
readiness for students, schools, districts and states (using each 
metric alone, and combined in model). 

• Three performance levels of College Ready (not CR, CR, CR Plus). 

3. Reporting - Beginning in late summer of 2009, we hope to 
release a college readiness report to the nation with state level 
information, as well. 

4. Begin providing diagnostic-type information - When students do 
NOT meet the college readiness levels, we will provide reports 
trying to understand some reasons for this and (at some point in 
the future) offer suggestions for improvement. 
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Some thoughts on Reporting 


At the end of the day - we will roll 
out reports that illustrate: 

• The number and % of students at 
each Academic Intensity level 

• The number and % of students 
reaching the SAT benchmark 

• The number and % of students 
reaching an HSGPA benchmark 
(2.7, 3.0, 3.5) 

• The number and % reaching 2 
and all 3 benchmarks 

• The probability of college 
success for each of these marks 
in terms of (completing freshmen 
year, FGPA of B- or higher, no 
remedial courses) for individual 
students, subgroups, schools 
and states. 


Fictitious Data 


Benchmarks 

(Example) 

Your 

Sch.% 

State 

% 

US 

% 

Prob. Of 

College 

success 

HSGPA 

(3.0) 

85 

82 

83 

61 

SAT 

(1180) 

30 

26 

29 

71 

Acad. 

Intensity 

(11-12) 

40 

39 

38 

72 

All Three 

21 

16 

22 

78 
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Additional Work 


> The College Board has developed two instruments that measure 
non-cognitive predictors. 

■ We are piloting these with 16 post-secondary institutions 

■ We hope these offer information about students in other areas. 

> We will examine relationships and establish benchmarks using 
these college readiness indicators with other types of college 
outcomes: 

■ Cumulative GPA 

■ Grades in specific disciplines 

■ Retention and graduation 

> Once reports are delivered, the College Board will examine how 
information is used by educators (Fall 2009) 
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Questions, Comments, Suggestions 


• Researchers are encouraged to freely express their 
professional judgment. Therefore, points of view or 
opinions stated in College Board presentations do not 
necessarily represent official College Board position or 
policy. 

• Please forward any questions, comments, and 
suggestions to: Thanos Patelis tpatelis@colleqeboard.org 
or 21 2-649-8435 

• Please go the College Board’s web site for much more 
information: www.co leqeboard.com/reserach . 


Thank you!! 
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